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ABSTRACT 


Any large procurement is of necessity a multi-criteria decision. The military 
acquisition decisions are typically complicated by military requirements and political 
considerations. A Decision Support System (DSS) can play an important role in helping 
military decision makers to come to better acquisition decisions. This thesis introduces the 
current system of military acquisition used 1n the Republic of China Navy and demonstrates 
a small DSS for assisting higher level managers in making acquisition decisions. A survey 
of ROC Navy Officers at applicable levels of the procurement system was taken to determine 
the criteria to be modeled in the DSS. This criteria were weighted using typical statistical 
methods. The results of the survey were used to construct a model for the decision to 
purchase a fictitious weapons systems. The model was extended for the purpose of the thesis 
to create a more realistic list of criteria used in a typical weapons system acquisition. An 
example, software system (Criterium software) was used to simulate the model and the 
software was exercised to demonstrate the interactive nature of the DSS. The alternative 
selected by the software was in accordance with experience and a direct reflection of the 


results of the survey. 
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I. INTRODUCTION 


A. MOTIVATION 


The advances in computer technology in the last decade have made available the 
tools to accomplish certain missions that before that time would have required much 
greater involvement of programmers, technicians and other people with specialized 
knowledge. These advances have also made the solution to some problems more 
economically viable. Decision Support Systems (DSS) have become more useful and 
accessible to managers because of the wider availability of more economical hardware 
and software. The computer’s ability to organize and present data for a manager to 
analyze and the increase in the capacity of desktop computers, make them an ideal tool 
and/or element in a Decision Support System. 

This thesis will address DSS as used in a large-scale procurement decisionmaking 
process. It will address military procurement and the factors and information that the DSS 
should provide to the decisionmaker. Some of these factors are costs, availability, 
capabilities, maintainability and other factors which could be considered political. To be 
useful the DSS should have in-depth and broad capability to present the results of data 
collection, budget information and other input variables that the decisionmaker would in 
fact use to make these procurement decisions manually. 

The decisionmaking process in any large-scale organization has many factors that 
may directly or indirectly affect any given decision. Some factors which in reality should 
not effect the decision may nevertheless be used in the decisionmaking process and lead 
to a bad or costly decision. This process on the surface may appear to be simple. In 
reality, it is very complicated and a decisionmaker cannot be expected to reach a perfect 
decision. Complicating this are the normal human factors in any organization: negligence, 
simple mistakes, or malfeasance. These factors could lead to loss of money or property, 


endanger human life, or seriously undermine the nation’s ability to defend itself. 


A goal of any procurement system is to reduce the possibility of mistaken 
decisions or errors in procurement. I believe a DSS would be useful to the Republic of 
China, to reduce costs and help decisionmakers make more accurate and timely 
procurement decisions. 

The Republic of China has reached a point in time where it is necessary to replace 
or upgrade obsolete military equipment. The international situation and the needs of 
Republic of China require this process to be accelerated if we are to accomplish our goals 
of self-reliance and increase our combat capabilities. A sound procurement system based 
on accurate data collection and a DSS would serve as a much better basis for procurement 
than procedures used in the recent past. 

The nature of military procurement systems is that decisions are more likely to be 
made in the political sphere than in the more practical sphere of military requirements. 
This process could ultimately lead to the Republic of China being unable to defend itself. 
The establishment of a DSS would lend to a procurement system as an institutionalized 
decisionmaking process, less influenced by the political process and more resistant to 
fraud and other abuses which lead to these scandals. 

The quality of decisionmaking would also be improved with a DSS in that the 
system would present all the factors needed by the decisionmaker in any procurement 
decision. To present these factors an effective data collection system is necessary. This 


data collection system itself would minimize bad decisions. 
B. BACKGROUND 


I have spent a great deal of time in the study of procurement regulations and other 
documents related to procurement. The military procurement system is Byzantine in 
nature, changes from moment to moment and defies any attempt at simplification. I have 
also interviewed staff personnel involved in military procurement. The purpose of this 
study and interviews was to identify the problems and factors which should be presented 


by any DSS system. The following are initial findings: 


Procurement is a Multi-Criteria Issue. Factors such as cost, longer range, 
greater capabilities, or easier maintenance, must be considered together to 
reach a decision. This means that a single factor, cost for instance, cannot be 
used to make the decision. 


Dissimilar Selection Criteria. Sound procurement decisions rest on a wide 
variety of often dissimilar criteria, including various cost and performance 
criteria, risks, political interests and a number of support criteria affected by 
all of the above. The criteria falls into two basic categories: qualitative and 
quantitative. Quantitative criteria, such as the various costs, can often be 
directly compared with one another. However, dissimilar quantitative and 
qualitative criteria, such as procurement cost and degree of risk, cannot be 
easily compared. A DSS can help address this problem of reconciling 
dissimilar criteria. 


Procurement Time. Each procurement regardless of size requires a certain 
minimum of time for decisionmaking and decision implementation. This time 
is required for research and study of the factors and the effects of the decision. 


Political Consideration. The political situation 1s more complex than ever 
before. It is difficult to find a decision that can be satisfactory to all parties. 
Unnecessary complications and limitations are generated, and even worse, 
some political scandals emerge, such as corruption, hidden agendas and 
favoritism. 


Conflicting Interests. In large systems there are many stakeholders and it is 
inevitable that there will be conflicting interests. It may be impossible to 
equitably balance the interests of every stakeholder. 


Subjectivity. The objectivity of procurement personnel 1s influenced by their 
experience and position in the system. Their thinking 1s subject to factors of 
career and acceptability in the group. Their decisionmaking process cannot be 
normalized without the help of an institutionalized system of decisionmaking. 
Military personnel, in particular, are subject to changes in status and position 
which will have varying effects on their morale and their ability to make 
objective decisions with regard to procurement. 


Time Constraints. A supervisor’s time is usually filled with urgent problems 
and this leads to the syndrome of “fighting fires”. The inefficiency of this 
approach leads to the neglect of the long-term procurement process and results 
in inefficiency and higher cost. The supervisor’s decisionmaking is unduly 
affected by whatever fire he may currently be working on. Significant errors 
cannot be avoided because of the supervisor’s short horizon. He 1s unlikely to 


see far enough into the future to predict them. Nor is he likely to have the time 
in study to make efficient and correct or timely decisions. 


e Day-to-Day Process. Data collection and analyses in the procurement 
processes are done repeatedly. New data require new analyses. The repetitive 
nature of these operations leads to errors and lackadaisical performance by 
personnel involved in this task. These same personnel are subject to the 
feeling that a task that is done over and over again cannot possibly be of any 
importance, since it will again have to be done next week or even tomorrow. 
This will lead, in time, to an accumulation of errors that will make the data 
collection worthless. 


C: WHY USE A DECISION SUPPORT SYSTEM? 


A successful procurement decision depends on the information available to and 
influences on the decisionmaker. A good decision will improve the capabilities of the 
military and/or save taxpayer’s money. A bad decision may result in inefficiency and the 
reduced effectiveness of the Republic of China’s combat forces or in a worse case 
scenario the failure of those forces in combat. These problems should be studied very 
carefully and the decisions should have the best support systems available. If a DSS can 
be developed for the Republic of China, it would provide decisionmakers at different 
levels with better information to base their decisions. 

Computer capabilities and tools to accomplish our mission of improving the 
procurement process are readily available. These tools would be used even if the DSS 
system were a manual system; therefore, it would improve the process if a standard 
computer system, hardware and software, were made a part of the design of the DSS. 

I believe a computer base DSS would be the best solution for our military 
procurement problems. It would improve the quality of decisionmaking, providing the 
best recommendations for procurement. In Chapter II, I will cover in detail the type of 


system and software that would meet the Republic of China’s requirements. 


D. METHODOLOGY 


Shown in Figure 1 is a flow chart depicting the design of a suitable DSS for 


military procurement decisionmaking problems. The major steps include the following: 


DEFINITION 


ANALYSIS 


QUESTIONNAIRE 
DESIGN 


DECISION CRITERIA 


ASSIGNING 
WEIGHTS 


SIMULATION 


CONCLUSION 


; 
cS 


MMENDATION 


Figure 1. Design Process of DSS 


e Definition - Definition of the problem. The definitions of the goals we expect 
to achieve and the definitions of the factors important to the decision. 


e Analysis - The characteristics of the problem are systematically analyzed for 
their influences on the decision. The most important characteristics are used to 
develop the questionnaire. 


e Questionnaire Design - The questionnaire is designed with the results of our 
definitions and analysis in mind. It should also reflect relevant questions for 
the stakeholders involved in the decision. 


e Decision Criteria - The results of our questionnaire are used as a source of 
statistical data to develop the system. With these results weights are assigned 
to the various factors. This should produce a prioritized list of factors from 
most important to least important. This list is used to capture the most 
important criteria in the decision model. 


e Assigning Weights - Using the results from the statistics package, assign 
weights to the various factors that are representative of the results from the 
user’s questionnaire. 


e Simulation - Choose a small-scale procurement problem as an example and 
perform a test with the software and the weighted criteria. 


e Conclusion - Determine the effectiveness of the DSS by interviewing a user of 
the system and weighing the results against the users criteria and probable 
decision. The interview should also include the users opinions as to the 
usefulness of the system for making decisions. 


e Recommendation - Recommend expansion of the system or modification of it 
as a result of the conclusions reached in the previous step. 


E. SCOPE AND LIMITATIONS OF RESEARCH 


Military procurement is a dynamic system. It is impossible to design a perfect 
solution that will be applicable in any particular scenario. My intent in this thesis is to 
provide some solutions that will be helpful in making procurement decisions in a limited 
scope of problems. For a general solution to these problems, a great deal more study 


would be required. 


Many elements of military procurement are classified. It is therefore difficult to 
collect actual data which could be used in my demonstration. As a result, I have created 
data that reflects the data from real procurement scenarios, in order to avoid the issues of 
classification, clearances, and confidentiality. Although the data I use is not confidential, 
I believe it represents a normal procurement and will demonstrate the concept outlined in 
the thesis. In Chapter II of this thesis, I will discuss the theory of Decision Support 
Systems and their structure and how they differ from expert systems. I will also review 
some examples of Decision Support Systems that have been used in civilian and military 
procurement. In Chapter III, I will discuss the design of the DSS that I will demonstrate 
in this thesis. Chapter III will consist of step-by-step procedures and the design decisions 
made for each step shown in Figure 1. Chapter IV will be an explanation of the results of 
the simulated procurement problem using the DSS designed in Chapter III. In Chapter V I 


will present my conclusions and recommendations. 





Il. DECISION SUPPORT SYSTEM STRUCTURE 


A. THE DEVELOPMENT OF DSS 


In the early 1950’s the computers use was restricted to scientific and engineering 
uses. However, improvements in technology and availability moved the computer from 
scientific uses to business uses. One of the first uses of the computer in the commercial 
world was for Transaction Processing Systems (TPS). TPS includes such things as 
payroll, record keeping and billing systems. Even with these uses, no one could have 
predicted the impact that computers would have on business and management at the 
present time. Speed, accuracy and storage capacity are the three characteristics of 
computers. Since that time, the computer’s capacity in all three of these categories has 
increased geometrically. These improvements are the very reason that computers are so 
useful as information processing devices. 

There are five uses for computers in management. The first, Transaction 
Processing Systems (TPS) is the most obvious and still the most common use of large 
computer systems. The next uses have more impact on management decisionmaking. 
Management Information Systems (MIS) are used for production control, sales forecast 
and monitoring these items. Decision Support Systems (DSS), the subject of this thesis, 
are used for long-range planning and complex decisionmaking. Expert Systems (ES) are 
used for diagnostics, internal controls, planning and maintenance areas that have a limited 
domain. The fifth use is as Executive Information Systems (EIS), these systems are used 
to support top management decisionmaking and provide top management with reports. 
[Ref. 1] 

The classification of these tools does not necessarily indicate that any particular 
computer system is dedicated to any one of these operations, or in fact, that any particular 


data source is dedicated to one management system. These technologies are made up of 


three dimensions: a particular computer/hardware, several programs/software and the 
management processes that gather information to support these systems. [Ref. 1] 


The development of these systems are as follows: 


e TPS was first used in the early 1950’s and is still the main use of large 
computer systems. 


e MIS was first used in the 1960’s by upper management mostly to produce 
standard operating procedures, decision rules and reduce cost. This was 
typically done by replacing clerical personnel who were employed to produce 
reports with a computer system. These systems were restricted to higher 
management for reasons of equipment cost as much as the utility of the MIS. 


e DSS. In the 1970’s Keen/Morton coined the term DSS [Ref. 2]. DSS is used 
to address semistructured problems and one of a kind or once in a lifetime 
decisionmaking situations. This differs from MIS which generally supports 
only recurring reports or highly structured situations. 


e ES were developed in the 1960’s as part of research in artificial intelligence 
(AI). Expert systems have a limited domain in that they are only applicable to 
recurring problems and are intended to assist or replace a human expert. These 
systems are not adaptable to new or unique situations. 


From the above list we have a basic knowledge of management’s support systems 
and we can easily distinguish between them. Figure 2 indicates the relationship between 
these systems. 

Figure 2 is a notional view of these relationships. This view shows the evolution 
of the use of data processing systems in decisionmaking, but does not represent the future 
development of management uses of computers. Figure 3 represents the theoretical view 


of DSS and the effects of information technology on an organization. 


B. WHAT IS DSS? 


54 


Decision Support Systems (DSS) are defined by Sprague and Carlson “... as 
interactive computer-based systems that help decisionmakers utilize data and models to 


solve unstructured problems” [Ref. 3]. The key points of this definition are interactive 


Decision Focus 


Focus 


Data 


a) lL) a 


Figure 2. The Notional View After Ref. [3] 


and unstructured. MIS is not usually interactive and the reports generated for the manager 
are usually structured and repetitive. Another important point is that the purpose is to help 
the decisionmaker to solve the problem. It is not to replace the decisionmaker as in an 
Expert System. Bennett’s definition of DSS: 


A coherent system of computer-based technology (hardware, software, 
and supporting documentation) used by managers as an aid to their 
decisionmaking in semistructured decision tasks. We stress supporting 
rather than replacing managerial judgments. We focus on improving the 
effectiveness of decisionmaking rather than on merely improving its 
efficiency. [Ref. 4] 


Bennett defines when a task is considered an unstructured task: 


e Objectives are ambiguous and nonoperational, or objectives are relatively 
operational but numerous and conflicting. 


MI) 


Management Level Dimension 





Interactive Models 
OR/MS/STATISTICS 


Functicnal Dimension 


Figure 3. The Theoretical View After Ref. [3] 


It is difficult to determine the cause (after the fact) of changes in decision 
outcomes and to predict (in advance) the effect on decision outcomes of the 
actions taken by the decisionmaker. 


It is uncertain what actions taken by the decisionmaker might affect decision 
outcomes. [Ref. 4] 


oF COMPONENTS OF A DSS 


There are three components of a DSS: 


Language System (LS) - This system is used by the user to interface with the 
DSS. It may include direct retrieval languages and computation languages. 
This allows the decisionmaker to express commands and statements, but at the 
same time, limits the decisionmaker to a finite number of expressions. 
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e Knowledge System (KS) - This system contains the knowledge of this 
decisionmakers problem domain. The knowledge must be organized and must 
be retrievable in a systematic manner. 


e Problem-Processing System (PPS) - The PPS is a system which understands 
the decisionmakers statements or commands and the representation of the 
knowledge in the KS. The PPS takes the relatively simple commands from the 
language system and processes them into the more complex operations of the 
retrieval system or the KS and the computations required for the DSS. [Ref. 5] 


D. THE CHARACTERISTICS OF A DSS 


Even though there are various definitions of a DSS, usually dependent upon the 
viewpoint and background of authors in the field, the characteristics of a DSS can be 


summarized as follows: 


e DSS assists decisionmakers to deal with multi-level problems structured and 
unstructured [Ref. 6]. 


e DSS supports the decisionmaker with an adaptive point of view. In this point, 
a DSS is better than a conventional MIS in that a MIS cannot adapt itself 
easily to new or unique situations. 


e A DSS 1s interactive. The decisionmaker can use the system to collect, 
process, display, store and retrieve information in real time. 


e A DSS is used to support the decisionmaker not replace the manager’s 
judgment and experience. 


e The purpose of a DSS is to improve the effectiveness of the decisionmaker 
and is not generally targeted at efficiency. 


e A DSS must be easy to operate because it is designed to be used by the 
decisionmaker not computer experts or even specially trained clerks. The 
system is interactive and is intended for the direct use of the decisionmaker. 


e The DSS must be adaptive. Over time, the type and nature of the decisions 
that the DSS is used for will change. If this were not the case, the DSS might 
better be replaced by an expert system. A DSS is intended to be used for one 
time only or unique decisions that are unstructured. 


[3 


A DSS must efficiently assist decisionmakers in making decisions. If the 
system is not efficient it simply will not be used by managers who must 
budget their time in the most effective manner to realize the goals of their 
organization. 


A DSS should assist in training inexperienced managers in that it will present 
to them information for a decision which their inexperience otherwise may 
have led them to overlook. 


E. THE FRAMEWORK OF A DSS 


The information used by the DSS is generated from interaction with the user, 


information from the database and model analysis. Figure 4 shows the components of a 


DSS which consists of five parts. 


1. 


Personnel 


There are three levels of DSS technology and five associated roles for managers 


and technicians in both the use and development of a DSS. They are shown in Figure 5: 


jeg 


Manager or user - The person faced with the responsibility of the decision. 
Intermediary - The assistant or the staff of the manager or user of the DSS. 


DSS Builder - The person who is familiar with computer systems and also 
familiar with the problem area of the decision. 


Technical Support - The person or a team who is acquainted with the problem 
area but whose expertise is in database, management, model building and the 
computer system that supports the DSS. 


Toolsmith - A person or a team whose responsibility is to develop new 
technologies, software and hardware to provide the DSS with better or more 
complete data or models. [Ref. 6] 


Hardware 


Computers, peripheral equipment and facilities for the maintenance and use of this 


equipment are required. Telecommunication equipment for remote connections to 


databases and other information sources are needed for large systems. Hardware to 


maintain backups of the software and databases used by the system are required except 
for the simplest systems. Remote storage facilities to maintain archives of the systems 
software are necessary for safety. 

3. Software 

Software consists of Database Management Software (DBMS), Model Base 
Management Software (MBMS) and dialogue generators for the interactive element of 
the DSS; compilers and special software used to maintain the system and the networks 


used by the system; software for maintaining archives of databases and program sources 
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Figure 4. Components of a DSS After Ref. [6] 
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Figure 5. DSS Levels and Associated Roles From Ref. [6] 


used in the system and software for debugging the DSS; and diagnostic software for the 
maintenance of the systems computers and telecommunication facilities. 

Software makes up three subsystems of the DSS, the dialogue subsystem which is 
used for interacting with the user, the data subsystem which supplies the DSS knowledge 
system and the model subsystem software which is used by the PPS in calculations and 
the presentation of data to the user. Peripheral to this, are the normal maintenance 
software used to maintain the data system and the hardware of the DSS. 

4. Database 

The database used by the system is a collection of information necessary for the 


DSS to function. It must include software and hardware required to maintain any database 
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system. This includes backups and conversion software to present the database in a usable 
form to the DSS. It may include software required to access remote databases across 
networks or other telecommunication links. 

3 Model Base 

The model base is made up of standard mathematics and statistic packages used 
by the DSS. It may include special purpose software created to support a particular 


decision process. 
Fe DIALOGUE SUBSYSTEM 


Much of the power, flexibility and usability characteristics of a DSS are derived 
from its interface to the user. This makes the dialogue subsystem software the most 
important subsystem in the DSS. Without a flexible and very usable interface, the DSS 
will not be used by managers who cannot devote time to overcoming limitations in the 


user interface. The dialogue subsystem is itself made up of three systems: 


e The Action Language - The software which interprets the users input and 
conveys commands or requests to the DSS. 


e The Display Language - The software which displays the results of requests, 
commands and model runs to the user. This may be in the form of a CRT 
screen, printers or other graphics output. 


e Knowledge Base - Knowledge base consists of the organizational knowledge 
and the users previous inputs. This may include manuals and help files for the 
operation of the DSS. [Ref. 4] 


Figure 6 shows a typical dialogue system. 


I 












User Goal Program 






-Presentation 


i Language 
(= ERE AQ. Mg eas Tr anslate 


‘ 





Thought 






for Display 









Action Plan 


Compute 







Translate 





for System 





Data Access 





banguage 





Figure 6. User/Terminal Interface After Ref. [4] 


G. THE DATA SUBSYSTEM 


The functions of the data subsystem are to query the database as a result of 
requests from the models and dialogue subsystem. This should include the ability to 
maintain the database records by inserting, deleting and adding individual records. This 
system should, in addition, be able to request updates to databases that are held locally at 
the users site so these databases reflect data whose original source is from remote 
systems. The data subsystem is shown in Figure 7 

1. Rich Set of Data Sources 

The data for the DSS must come from external sources and cannot depend solely 
on local or internal sources since the decisions made by upper management levels are 


heavily dependent on external data sources. In addition, management decisions must also 
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Figure 7. The Data Subsystem From Ref. [3] 


be based on decisions made in the past. This requires local historical data be presented by 
the DSS. 

2 Data Capture and Extraction Process 

The nature of the DSS requires that the data extraction process and the DBMS 
which manages the database be flexible and allow rapid response to users request for 
data. This is because of the interactive nature of the DSS. If a manager spends too much 
time waiting for a request to be satisfied by the DBMS, the next time he will not make the 
request, thereby, bypassing that piece of information in making the decision and 


essentially disregarding the DSS in his decisionmaking process. If the system is not 
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flexible the manager may not even be able to request the information he requires to make 


the decision, which will again result in the DSS not being utilized. 
H. THE MODEL SUBSYSTEM 


The most promising aspect of DSS is its ability to integrate data access with 
decision models. It does this by imbedding the model subsystem in the DSS and 
providing a database on which to operate the models. This integration provides powerful 
what-if scenarios for the user. Figure 8 shows the components of the model subsystem. 


[Ref. 3] 
The key capabilities provided to the DSS by the model subsystem include: 


e The ability to create new models rapidly and easily. 


e The ability to catalogue and maintain archives of a wide range of models 
supporting all levels of management. 


e The ability to access and integrate models to create other models and use these 
as building blocks to more complex simulations. 


e The ability to interrelate these models with the database. 


e The ability to manage, maintain and archive these models as if they were 
records in a database. 


I. THE DEVELOPMENT APPROACH FOR DSS 


There is no universal decisionmaking theory; varying conditions and rapidly 
changing circumstance make a DSS a good tool for tracking change. In order to track 
changing circumstances an interactive system combining analysis, modeling, data access 
and presentation in a single step is required. An initial system can be built to solve a 
small problem. Once this system is capable of supporting decisions in a limited area, it 
can be expanded and improved until it will support a wide variety of decisions required 


by the organization. DSS are in a constant state of change reflecting new technology, new 
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data sources and new management strategies. This in itself should lead to a system which 
is adaptive and flexible. 

An adaptive system is defined as a system with three abilities related to time. In 
the short term it must be flexible enough to solve a given problem within a given scope. 
In the midterm, the system must evolve to accommodate changes 1n scope and in the long 


term, it must accommodate itself in any given situation. 
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Figure 8. The Models Subsystem From Ref. [3] 
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Ill. DSS APPLICATION DESIGN 


This chapter will cover my design of a DSS to support the decision to acquire a 
fictional weapons system from either a domestic supplier in the Republic of China or 
from a group of international suppliers. Decisions of this kind involve cost, political, 
technology transfer and support factors. Decisions to acquire a system that are based on 
these factors are made at very high levels of government. At this level of government, a 
decision maker can demand very detailed and extensive information from the bureaucracy 
below. My model of a DSS is very simple and includes very few criteria included in the 
decisionmaking system, but nevertheless, I believe it is a very valid system because at 
this level, decision makers need to reduce the detail involved in a decision down to 
factors which will make their decision understandable and in fact, make it possible to 
come to a decision. So the criteria I have selected for a decision at this level represents a 


large collection of criteria. 
A. DESIGN CONSIDERATIONS 


The first step in this chapter will be to select a DSS that has the following 


characteristics: 


e Simple to Use - This system will be designed for very high level users who do 
not have the time to learn obscure user interfaces or interpret computer output 
which cannot be grasped immediately. For people at this level, the best system 
and the most likely to be used system is one that has a very simple user 
interface and presents to them easily understood results. In modern computer 
terms that would be a graphical user interface where input is performed by 
clicking and dragging objects with a mouse. The normal output from these 
interfaces are usually colored graphics. Graphics have the capability of 
presenting large amounts of information almost instantaneously. 


e Flexibility - The nature of a DSS is that it is flexible. It is required to present 
results in real time to the user so that the user can make a decision. Different 
users will require different criteria or weights to that criteria and may require 
different presentations of the results. The key ingredient of a DSS is that it is 
intended to provide information to the user so that the user can make a 
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decision. Without flexibility the DSS may not be able to present data to the 
user in a form he requires or adjust itself to new decision criteria that will 
naturally evolve over time. In addition, a DSS is designed to assist in making 
a decision to a unique problem. Therefore, if it was inflexible, it would not be 
used more than once. 


e Quick Results - A system which is particularly slow in displaying results may 
be used the first time by a busy manager. However, the loss of time and the 
period of idleness represented by it will be considered when he is deciding to 
use it again. Depending on how slow this system actually is, he may not use it 
at all, and invest that idle time in more traditional methods of making a 
decision. This problem would probably not show up on a small system that 
does not depend on queries to large and/or remote databases. If the DSS and 
its data can be centrally contained in a small computer, quick results can be 
expected. Decisions to buy floating point units or simply better software 
would be expected to overcome problems associated with speed. Quick results 
are also necessary for a truly interactive system. If the user is not presented 
with immediate feedback, he will not be able to rapidly test scenarios and an 
important feature of a DSS used in generating information for the user will be 
lost. 


e The Ability to Quantify Criteria - The system must be able to represent criteria 
as a number. For some criteria this would be a simple process, but usually this 
is only true of factors involving cost. This number scale must also effectively 
represent quantities that are less tangible, for example, the effectiveness of a 
weapons system, the political cost or risk factors. 


e Interactive - One of the key ways for the user to simulate the effects of his 
decision is to modify the various factors until an acceptable result 1s achieved. 
This process begins when the user changes the model, examines the results, 
and uses those results to make improvements to the model. This process can 
also be used to discover the critical points in the criteria which would result in 
a different decision being made. This feature will generate better decisions 
over time as the user gains a better understanding of the criteria or factors that 
affect his decisions. Without an interactive feature, the decision maker would 
be presented with results which he would not understand and in all 
probability, lose confidence 1n over time. 


The software package selected for this demonstration system is Criterium 
Decision Plus made by Sygenex. This software runs under the Windows Operating 


System and it meets the design considerations listed above. It is very easy to use, and has 
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a graphics interface. It can use one of two models, Analytical Hierarchy Process (AHP) or 
a Simple Multiattribute Rating Technique (SMART). It has output capabilities which will 
show the user the most critical factors in the decision and show the points at which the 
alternatives will change. The hardware requirements for the program are very ordinary 
and can be met by any recently purchased PC. The system includes a brainstorming 
feature which leads to a rapid design for the DSS. The software includes more advanced 
capabilities which I have not used in this demonstration. They include an uncertainty 
capability and the ability to generate reports. 


This software operates on the following three principles: 


e Hierarchy Representation - The problem is divided into definable elements. 
e Priority Discrimination - Elements are ranked relative to one another. 
e Synthesis - Individual judgments are combined into an overall rating. 


Criterium software allows a comparison of each combination of alternatives for each 
criteria and allows the user to assign weights to each criteria. This software is extremely 
useful in organizing and prioritizing multiple alternatives and multiple criteria. The 
software performs a pairwise comparison between criteria and then selects the best 
alternative for the user. 

This software could also be used in a Group Decision Support System (GDSS) but 
it would require good leadership and management skills to bring various stakeholders 
into the process. The software has no automatic communication features (electronic mail) 


nor does the software provide for database queries normally associated with GDSS. 
B. DESIGN METHODOLOGY 


For the design of the DSS, we did not have access to the Taiwan Navy database. 
In order to make our system represent a real system, we have developed a questionnaire 


and generated data that would represent this database. 
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Jt Definitions 

The initial step in the design of the DSS is gathering definitions of criteria and the 
problem domain. The problem domain is to decide which one of four fictitious weapons 
systems to purchase. System A is a weapons system produced by the French, System B is 
produced by Korea, System C by the United States and weapons system D is produced by 
the Republic of China. These systems all have different capabilities, costs and political 
considerations. 

The criteria used to make the decision was set to the feasibility of acquiring the 
equipment, the cost of the equipment, delivery schedule, political factors, the technology 
transfer to be gained by the ROC, the feasibility of acquiring the weapons system and of 
course, the capabilities of the weapons system. This is a short list of criteria to be 
considered when purchasing a weapons system. A more complete list would include 
detailed factors involved in cost and equipment capabilities. The factors of politics and 
technology transfer probably cannot be further detailed simply because they are subject to 
the decision maker and would represent his opinion. Nevertheless, the initial factors were 
broken down into subfactors which represent a more realistic level of detail. The 
subfactors were given decision level data not based on the survey, but based on my 
experience in purchasing weapons systems. The definitions for all the criteria used in the 
model are listed in Appendix B. 

With the Critertum software the first step is brainstorming which is quite easily 
done by simply defining a goal for the DSS and creating a block for each of the criteria 
listed above without regard to the structure of the decision. The brainstorming window of 
the Criterium software is shown in Figure 9. 

J Analysis 

The second step is the analysis of the criteria selected in the first step. This 
requires us to determine the relationship or hierarchy of the criteria in relation to our goal 
or decision domain. The analysis of the criteria that make up the decision results in six 


groups, they are: 
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Figure 9. Brainstorming Window 


e Technology transfer which is the only item in its group. This criteria indicates 
the technical capacity which will exist after the procurement of the weapons 
system. This criteria represents an advantage for domestic development and 
research. 


e Combat capability, which is the only item in its group. This criteria represents 
the performance characteristics of the weapons system evaluated on the same 
numeric scale as other criteria. It is a composite of the capabilities of the 
weapons system. In a more realistic DSS this group would be a sum of the 
many subgroups representing the detailed capabilities of the system. 


e Cost, which is made up of two subgroups. The initial purchase price of the 
weapons system and its lifetime support or logistics support and the 
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operational cost of the weapons system or operational support. Logistics 
support represents part support for maintenance of the system, the reliability 
of supply (domestic or international), and the cost of tooling up for parts that 
have become obsolete. Operational support is the cost of supporting the 
weapons system when it is in use. This includes training, maintenance and the 
personnel to operate and perform these functions. This cost in a real world 
system would have to reflect increases in personnel for maintenance, 
operation, training and the effects of the weapons systems space and weight 
requirements on board a vessel. 


e Risk, which is made up of political considerations and the delivery time of the 
weapons system. Certainly in a real world system there would be more 
detailed criteria contributing to the risk criteria, for example, early 
obsolescence of the system, unsuitability for shipboard use, low reliability, 
does not meet the specifications as advertised, cost overruns, and/or the 
possibility of non-delivery. 


e Feasibility of weapons acquisition, which is the international political 
considerations involved in acquiring a weapons system. 


e Political considerations, which is the effect of political considerations both 
domestic and international on the selection of a weapons system. For example, 
in the international category political considerations would include 
internationally recognized embargoes or acquisitions of weapons systems 
which may initiate an arms race with a neighboring country. Domestic 
considerations would include high government level strategies for purchasing 
the weapons systems or the wishes of the legislative body for either domestic 
purchase or foreign purchase of the weapons system. 


The results of the analysis are shown in Figure 10. 

3. Questionnaire Design 

Military procurement is a complicated issue. A questionnaire to survey every 
aspect of military procurement would be prohibitively large. Such a survey instrument 
would be unlikely to gain an adequate response or to be willingly completed. I have 
designed a questionnaire to obtain the criteria needed for a simple decision hierarchy. It 
includes eight questions with the possibility of the respondent filling out an additional 


seven questions. The questionnaire is short, but the questions have been designed such 
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Figure 10. Results of Analysis 


that their scope will provide the information needed to design the DSS. Additionally, 
some of the questions were designed to solicit information outside the scope of the DSS, 
but indicate the willingness of the respondent to use a DSS. Several questions are 
designed to see if the respondent is familiar with the term DSS. 

When designing a questionnaire for the development of the DSS, it is necessary to 
develop questions which will return measurements for the various factors or criteria that 
are required for the operation of the model developed in the analysis step. The survey 


questions are in Appendix A. Question 3, which has seven optional questions, and 
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Question 4 are the questions which will be used to generate the weights for the criteria 
developed in the previous section. 

Questions 3 and 4 only support the generation of criteria at the second level of the 
hierarchy, with the first level being the goal level. The hierarchy is shown in Figure 11 
and detailed information can be found in Appendix C. The other levels of the hierarchy 
are based on my experience and are intended to show a more realistic hierarchy for the 
decision. For this thesis I could not expect to have the organizational backing for a more 
extensive survey that would provide the depth of information required for the actual 
factors at lower levels in the decision hierarchy. Therefore, I limited my survey to the 


first level of the decision hierarchy. 






Weapons System Selection 


Figure 11. Decision Hierarchy 
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The other questions in the survey are intended to create a common vocabulary and 
direct the respondent to the subject of the survey. Naturally, this cannot be an extensive 
survey. My respondents are senior officers in transit between commands, going to 
military schools in the ROC. 

The respondents to the questionnaire are at different levels of the procurement 
system. Since this system is designed for very high level management, I have weighted 
the responses from the questionnaire in a ratio of 3:2:1. The ratio 3 is for respondents 
who are the actual decision makers. These are high ranking executives usually in 
headquarters commands. The ratio of 2 is for staff and other commands that make 
recommendations directly to the decision maker. The ratio of 1 is for the user of the 
weapons system at a level of commanding officer and leaders who may be directly 
responsible or involved in the equipment’s use. Each of these respondents have a 
different view of the purchase of a weapons system. For example, the user of the system 
is usually overly concerned with the capabilities of the weapon. The staff concerns are 
directed towards planning and support for systems throughout the fleet. The decision 
maker is concerned with the political and budgetary concerns and must look to the long 
range planning for the organization. 

The method of the survey included a recorded tape with a background of DSS and 
an explanation of the survey. Respondents who were studying in the National Defense 
University were surveyed collectively. Some other respondents were contacted 
individually. I sent out 262 copies of the questionnaire. I received 179 copies. From 
those, I randomly picked 100 copies for statistical analysis. The purpose was to reduce 
the mathematics involved in calculating statistics of the results of the survey. 

To be properly done, a survey for the development of a DSS should attempt to 
identify all criteria used by decision makers in the domain that the DSS 1s intended to 
function. I have provided in the questionnaire two questions which can be filled out by 


the respondent if the criteria listed is inadequate. 
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4. Decisions of Criteria 

From the results of the statistics obtained from the questionnaire, the importance 
of various criteria on a decision can be determined. The quality of output from the DSS is 
directly related to how well the criteria represent the decision process of the user. This 
step is the most important step in the design process. If there is some criteria not covered 
by the DSS or the definition of a criteria is not precise, the selection of the criteria will 
produce a system that is unsound or produces dubious results. To increase the credibility 
of my system and the likelihood that it will produce quality information, I have 


performed three steps: 


e Consulted with decision making staffs to acquire the factors or criteria that 
they consider important. Discussed with them actual example weapons 
procurement. 


e Taken into account the regulations for procurement in the ROC and related 
information and incorporated it into the demonstration DSS. 


e From the statistic results of the questionnaire, selected the most significant 
eight factors which will be incorporated into the DSS model. 


The results of the survey indicate that most respondents agree that the six factors 
selected are important considerations when making a purchase of equipment. When 
compiling statistics to generate the actual factors for each of these criteria, I selected at 
random only 100 of the responses to simplify the process of computing the statistics. 
Question 3 was used to determine whether the respondents agreed that these were 
important criteria. Question 4 was used to generate the weights that the respondents 


believed these criteria should have. The six criteria targeted in the questionnaire are: 


e Technology Transfer. The technical capacity which will exist domestically 
after the procurement of the weapons system. The greatest advantage is 
represented by a purely domestic development and research followed by the 
relative willingness of the source country to supply technical expertise and 
information. 
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e Delivery Time. The time required to deliver an operational system, and 
possibly the time to train personnel in operation and maintenance of the 
system. 


e Combat Capability. Represents the performance characteristics of the weapons 
system evaluated on the same numeric scale as the other criteria. This is a 
composite of the capabilities of the weapons system. 


e Feasibility of Weapons/Equipment Acquisition. The feasibility of acquiring 
the weapons system, this factor involves trade restrictions, international 
embargoes, and domestic budget considerations. 


e Political Considerations. The sum of all political considerations both domestic 
and international. 


e Cost. The cost of both purchasing the system and operating it. 


5: Assigning Weights 

The most significant problem in assigning weights is assigning the weights to 
non-numeric criteria. For example, the only numeric criteria available in the list of six 
first level criteria is cost. In the design of my questionnaire in Question 4 the respondents 
responded to the importance of each factor as a percentage of all factors. Additionally, 
they responded to the combat capability subcriteria in the same way. For example, 
combat capability equals 100% and the subfactors are a portion of this 100%. I used this 
response to generate a factor that represents the importance of each item in the decision. 
Further, I gave more weight to the actual decision makers response in the ratio of 3:2:1 as 
mentioned in the previous section. The sum of these multipliers and the response was 
used to generate an average weighting from the survey response. Since J] have no actual 
cost data to generate a numerical criteria, I performed the same operations for cost as the 
other five factors. Appendix C is a spreadsheet listing of the factors used in the model. 
The only factors computed from the survey were the six listed previously. The remaining 
factors in the model were generated ad hoc using my personal experiences and | believe 


they would represent the criteria at that point in the model. 
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The two most significant factors in the decision from these calculated results are 
technology transfer and the combat capability of the system. All other factors have 
approximately the same weight in the decision making process 

Technology Transfer calculated to a weight of .22. This weight is entered into the 
Criterium software as 22. All ranges in the Criterium software are from 0 to 100. In the 


Decision Hierarchy technology transfer consists of: 


e Knowledge and skills. Intangible skills and knowledge gained with the use 
and employment of advanced or new technologies. 


e Hardware and software. The actual equipment or software gained by the ROC. 


In this section of the hierarchy knowledge and skills was assigned a value of 60 and 
hardware and software transfer was assigned a value of 40. The alternatives are each 
assigned a value which is representative of that source for knowledge and skills and 
hardware and software transfer. The hierarchy of technology transfer is shown in Figure 


ee 
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Figure 12. Technology Transfer 
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Risk consists of: 


e Delivery Time. Calculated to a value of .11 from the survey results. This is 
entered into the hierarchy as 11 ina range of 0 to 100. 


e Technology Risk. Represents whether the weapons system is a finished 
technology or a leading edge technology. This points to the risk that the 
weapon may in fact be ineffective since it is untested in combat. 


Risk is set to a 60/40 ratio of delivery time to technology risk. Technology risk is made 
up of technology risk factors for each alternative. Delivery time was of course, a subject 
of the questionnaire and is set to 11. This is the only criterion that was subject to the 
questionnaire that was not at Level 2. The hierarchy for risk criteria is shown in Figure 


13. 


Risk Delivery Time 


Technology Risk 
Weapons/Equipment Acquisition * 


Figure 13. Risk 


Combat capability is calculated as .22 and its effect on the goal of the hierarchy 


was set to 22. Combat capability is made up of three criteria: 
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e Mission Effectiveness. A reflection of the effectiveness of the weapons 
system. 


e Integrated Logistics. Represents whether the technology of the weapons 
system fits well into the existing systems used by the armed forces of the 
ROC. A weapons which uses unique spares or supplies would require unique 
logistics to maintain and use. 


e Weapon Availability. The reliability and maintainability of the weapons 
system. 


Mission effectiveness is set to 80, integrated logistics to 5 and availability is set to 15 as 
to their effect on combat capability. The individual values for integrated logistics and 
availability are determined by the individual values for these factors from each 
alternative. 


Mission effectiveness is broken down into three subcriteria: 


e Range. The effective range of the weapon. 


e Integration. Whether the weapons system is well integrated with the ship's 
existing weapons system. Does it provide extra capability or duplicate existing 
weapons. 


e Payload Capability. Does the weapon have the capability of multiple types of 
warheads. Is the carrying capacity of the weapon suitable for the target. 


Range, integration and payload capacity are set to a ratio of 60:30:10 and each criteria 
receives a rating from the four alternatives. 

The hierarchy for combat capability criteria and its sub-criteria are shown in 
Figure 14. 


Feasibility of weapons acquisition is made up of three subcriteria: 


e Financial Worthiness of the Contractor is the ability of the company 
producing the weapons system to produce the system even though it may not 
be to their financial advantage to do so. It also indicates the companies 
technology expertise to research and ability to complete the project. 
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Mission Effectiveness 


: ———, } Combat Capability integrated Logistics 
Weapons/Equipment Acquisition 
Weapons Availability 


Payload Capability... 


Figure 14. Combat Capability 


e International Trade Policies. Some countries may have restrictions on the 
export of a particular type of weapons system, or there may be an 
internationally recognized embargo on the country that is the source for the 
weapon. 


e Domestic Budget Considerations is whether or not the legislative branch of 
government will budget money for this particular weapons system. 


Financial worthiness, international policies and budget considerations are set to a ratio of 
5:80:15 and each receives a factor from the four alternatives. Feasibility has a value of .17 
and is set to 17 for its effect on the overall decision. The hierarchy for feasibility of 
weapons acquisition is shown in Figure 15. 


Political considerations are made up of two factors: 


e Domestic political considerations include domestic politics involved with 
government contracting. 


34 


Financial Worthmess of the Contractor 
Feasibility of Weapons Acquisition International Trade Policies 


Domestic Budget Considerations 


Weapons/Equipment Acquisition 


Figure 15. Feasibility of Weapons Acquisition 


e International political considerations involve embargoes to outlawed countries 
or sources of supply which are not politically aligned with the ROC. 


Political considerations calculated to a value of 0.08 and are set to 8 in the model. There 
was small response to part G “Other”, that calculated to a value of .02. I included this 
response into political considerations rather than create a new criteria. Domestic and 
international political considerations are set at a ratio of 80:20. Figure 16 shows the 


hierarchy for political considerations. 






Weapons/E quipment Acquisition 







Domestic Political Considerations 
International Political Considerations 






Political Considerations 


Figure 16. Political Considerations 
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Cost consists of: 


e Operating Cost. The cost associated with the use of the equipment, for 
example, steaming time required to train with the equipment, the cost resulting 
from a training exercise which expends ammunition, missiles, etc. 


e Procurement Cost. Cost of the equipment at initial purchase including the 
initial maintenance of the equipment and the initial parts support for the 
equipment. 

Operating and procurement costs are set to an 80:20 ratio as to their effect on overall 


costs. 


Operating costs are further broken down: 


e Maintenance Spare Parts Cost is the cost of the parts to maintain the 
equipment and the cost of preventative maintenance spares to keep the 
equipment at readiness. 


e Training Cost is the cost of training maintenance personnel and operations 
personnel. 


Maintenance and training costs are set to a ratio of 80:20 and are represented in each 
alternative. 


Procurement costs are further broken down: 


e Research, Development and Engineering Cost is the cost of the development 
of the weapons system and the cost of its production. 


e Initial Procurement is the minimum cost of acquiring a production run of the 
equipment. 


e Initial Spare Parts and Training is the cost of initial parts support and the 
initial training of personnel to operate and maintain the system. 


e Technical Support is the cost of technical personnel to support the weapons 
system while ROC personnel are being trained. 


These costs are set in a ratio of 10:80:40:60 and are represented as individual factors for 


each alternative. The hierarchy for cost is shown in Figure 17. 
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Maintenance Spare Parts Cost & 


Weapons/E quipment Acquisition & 
Traning Got 


Procurement Cost 


Figure 17. Cost 


6. Simulation 

The Criterium software uses a pairwise comparison method to simulate the 
decision process. A pairwise comparison is made between each criteria in each group of 
the hierarchy to determine the effect of that group on the next level of the hierarchy. 

When the simulation is run with the criteria values entered in the previous section, 
it presents the weapons system produced by the Republic of China as the best alternative. 
The next best alternative is the system produced by the U.S. With the Criterium software, 
you can display a graph which shows the contribution of all criteria made to all the 
alternates. The magnitude of any alternate indicates its score and different colors indicate 
the contribution of each criteria to that score. In the same menu item, it is possible to 
display an ideal alternate which can be used to determine whether any alternate 
sufficiently meets the requirements of the decision. The results window for the Criterium 


software shown in Figure 18. 
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Figure 18. Results 


The user of the system will naturally question whether this is a reasonable 
alternative. The Criterium software provides provisions which will be covered in Chapter 
IV for analyzing the sensitivity by weights and the contnbution of various criteria. 
Additionally, not covered in Chapter IV, the Criterium software has provisions for 
tracking the uncertainty in the assignment of the weights to each criteria. 

a Recommendations 

The last step in our design methodology is to compile changes required to make 
the model perform correctly. There are two types of changes: the first type are changes to 


the weights of various criteria that are already in the model so that each level will reflect a 
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reasonable decision hierarchy. The second type are additions to the hierarchy or new 
connections between the levels of the hierarchy. These recommendations will be 
compiled after testing the model for reasonableness. Step 6 and 7 of our design 


methodology, shown in Chapter J, Figure 1, will be covered in more detail in Chapter IV. 
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IV. SIMULATION AND ANALYSIS 


In order for a DSS to be effective, the user has to understand the assumptions and 
criteria that underlie the model used in the system. Without an understanding of it, the 
manager could not trust the result nor could the manager modify the model to achieve 
better results. The first step in the process of achieving good results from a DSS is the 
initial design of the DSS. This design must represent the hierarchy of the decision. This 
was done in Chapter III. Although this design lacks sufficient detail for a real system, this 
detail was added by incorporating experiences in procurement. This created a DSS with 
enough sophistication that in this chapter will cover modifying this initial design into a 
system likely to be used in the real world in a real decision. 

A DSS is intended for use by the decision maker. In Chapter II, it was determined 
that the characteristics of a DSS require that it be easy to use and interactive. This chapter 
will demonstrate those aspects in the Criterium software. An understanding of the internal 
mechanisms of the DSS is not required for a user to modify the initial DSS created in 
Chapter III. If this kind of detail were needed by the user, this system would likely fail 
because the users at this level are only interested in the domain of the decision. These 
high level users cannot spend the time necessary in discovering the details of the software 
involved in the process. The software meets this requirement well by utilizing a graphics 
interface and displaying information about the process 1n a form easily understood by the 
user. The software provides tools so that the decision maker can modify the hierarchy and 


the weighting. 
A. REVIEWING THE RESULTS 


A DSS 1s intended to assist the manager make a decision. Although the manager 
could be the designer of the DSS, it is unlikely at this level of decision making (purchase 
weapons/equipment) that is the case. The best reviewer of the results of the model would 


in fact be the decision maker. In the weighting of the criteria for the initial model it was 
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necessary to multiply the effects of the survey described in Chapter III by a factor of 3. 
This was done because the number of manager’s surveyed at this level (high level 
decision makers), is very small and only they have a true feel for the political climate and 
long range goals of the ROC Navy. Others at a lower level tend to narrow their view to 
their own expertise and problems. This tends to make the lower level decision maker’s 
view less pragmatic than that of the person who must champion the actual decision. 

1. The Ideal Alternative 

The results of the decision are displayed by the DSS as a bar chart showing the 
relative score for each alternative. These alternatives are for fictitious weapons system A, 


B, C, D. Figure 19 shows the results of this initial model. The best alternative was a 
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Figure 19. Initial Results 
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weapons system produced and developed by the ROC. The DSS also displays an ideal 
alternative. This alternative is defined as the “perfect alternative”. This would be the 
alternative that had the highest weighting for any criteria. This is the simplest check on 
the reasonableness of the results. Compared against the ideal solution, we can ask 
ourselves the question of whether any solution compares well to the best solution. If the 
decision scores for the alternatives are a very low percentage of the ideal, it may indicate 
an unresolvable conflict in the criteria or tell the decision maker that he must look for 
better alternatives. 

Ze Contribution of Criteria 

One of the simplest analysis done by the software is a bar chart display of the 
contribution of each criteria to the decision. This display is compiled by level and shown 
in Figure 20 and Figure 21. The user selects the level, goal, level 2, etc. and displays the 
contribution to the goal or a particular criteria at that level. This gives the decision maker 
a view of the contribution at each level to the decision. With this information, the user 
can determine whether the contribution of a particular criteria has a reasonable 
magnitude. The graphics display at the goal level also shows the user which criteria have 
contributed most to the choice of the final alternative. This can be used to determine the 
source of any unreasonable information and would lead the user down the path of the 
hierarchy necessary to correct the model. The ability to select different levels allows the 
user to fine tune the model from level to level until the user discovers the source for the 
unreasonable behavior of the model. 

5: Sensitivity Analysis 

After the user has corrected the gross errors in the model using the contributions 
of various criteria and comparing the results to an ideal result, subtle errors will remain. 
These errors are the result of criteria in the mid-levels of the hierarchy that are sensitive 
to small changes in the previous level. In some cases, this may be an exact model of the 


decision process. In others, it may be a failure in setting the weights between various 
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criteria. For the user to analyze these effects, the software provides a sensitivity analysis 


mechanism. 
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Figure 20. Contribution to Goal 
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Figure 21. Contribution to Technology Transfer 


Sensitivity analysis is defined as changes in the weights or ratings that change the 
preferred alternative. The software provides a simple mechanism for the sensitivity 
analysis. The user selects the bar chart which shows the results of the model. Then the 
user selects the sensitivity analysis display and places them in such a way that the user 
can see both displays. The seaeaeia, display provides a pointer which the user can move, 
and simultaneously, the result chart will change to reflect changes in the criteria that the 


user 1s manipulating. 
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The sensitivity display lists the criteria from most sensitive to least sensitive. This 
is shown in Figure 22. In the graphics portion of the display where ever the lines for each 
alternative cross, at that particular point, the criteria will lead to a change in the displayed 
alternative. This mechanism is much simpler to operate than, for example, a system 
where numeric quantities are displayed. Simplified operations of this kind are a 
requirement for a DSS in order to be used by higher level managers. Its immediate 
feedback to the user gives the user the ability to make rapid tests to determine whether the 


model is a good representation of the decision process. 
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Figure 22. Sensitivity Analysis Window 
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Figure 23 shows two windows for sensitivity analysis and the results display. 


With the display setup in this manner, it is possible to see the effects of moving a 


particular criteria to a critical point and a change of the recommended alternative to the 
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Figure 23. Sensitivity Analysis and Results Window 


decision. The graphics would indicate when more than one alternative is involved in the 
change of the criteria’s value. This may indicate in the model that some criteria should 
not in fact be modeled because the range of this criteria is narrow for each alternative. 
This can normally be done by inspection. When the graph is displayed many of the 
alternative lines will cross the best alternative very near to the current value indicated by 
the cursor for that criteria. Any single alternative line crossing the best alternative may 
indicate that these alternatives are close together in value or it may indicate that one of 


the alternatives has been weighted incorrectly. Figure 24 shows the effect of moving the 
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line for knowledge and skills for the ROC alternative up to the critical point where the 
alternative choice changes from ROC to US. 

4. Tradeoffs 

This function of the Critertum software displays the criteria at the lowest level of 
the hierarchy as a ratio to a selected criteria. For example, if the user selects the criteria in 
the model of “Knowledge And Skills” as a reference, it shows a list where one unit of 
Knowledge And Skills equals X units of all the other lowest level criteria. This allows the 
decision maker to directly observe the reasonableness of the model by weighing the 
lowest level criteria against each other to insure that they have the proper relationship. 
This is the most difficult to grasp because it’s numerical in nature and is presented as a 


simple list. The Tradeoff of Display is shown in Figure 25. 
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MODIFICATION OF THE HIERARCHY MODEL 


Modification of the model is most easily performed by displaying the model 


hierarchy and double clicking on the block which represents the criteria the user is 


interested in modifying. This will bring up the window where original weights were 


entered. These weights are relative values between the criteria at the lowest level. These 


weights represent the relative worthiness of each weapons system in that criteria. The 


software automatically normalizes these numbers for the point in the level that the user is 


modifying. 
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1 Model Reasonableness 

The simplest way to determine whether the model is producing reasonable results 
is to examine the contribution by each criteria to the results. In Figure 20 is a display of 
the results from the current model. 

An examination of these results indicate that the largest factor contributing to the 
alternative of the ROC being selected over others, is technology transfer. This reflects the 
results of this survey which rated technology transfer and combat capability as equal 
importance and the two most important criteria in the model. This is also a reasonable 
assertion for the ROC in that technical capabilities gained by the ROC are important as 
they are important for any country that wishes to maintain a strong defense. Since this is 
the largest contributor to the selection of the ROC as the best alternative and the other 
alternatives appear graphically to represent the amount of technology transfer that will 
take place to the ROC, this would appear to be reasonable and unlikely to require 
modification. 

The next largest contributor to the selection of the ROC is combat capability. 
Combat capability appears to be equally distributed between the four alternatives with the 
exception of France which has a better combat capability. This is a reasonable result in 
line with my experiences in the procurement of weapons systems. 

All other criteria with the exception of risk contribute equally to the alternatives. 
Cost in the case of the Korean option is an advantage, however, the Korean option is the 
least likely because of its effects on technology transfer. This leaves the risk criteria to be 
considered. Each option has a different degree of risk associated with it. But this is 
unlikely to be modified because the ROC has the most risk associated with the selection 
and if the risk were moved to the highest value, it would still win out over the US option. 
A sensitivity analysis of risk indicates that this is the most likely factor to result in the US 
option being selected, but moving the ROC to a value of 0 (maximum risk) will only 


bring the ROC option equal to the US option. 
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The above analysis can be performed by selecting one display (contribution by 
criteria) and would assure the user that the results do indeed reflect a reasonable option. 
As an exercise I investigated the contributions to technology transfer from the next level 
of the hierarchy (Level 2). At this level the numbers used for weighting were generated 
by my experience and research into regulations and past procurements of these types of 
systems. 

Level 2 is shown in Figure 21. The contributions to technology transfer are two 
items, Knowledge and Skill and the Transfer of the Hardware and Software. Obviously, if 
you buy a weapons system, you will receive the physical aspects of it (hardware and 
software) but more important to the ROC is the technical knowledge and skills used to 
develop and maintain this equipment. This diagram again shows a realistic view of 
technology transfer. At this point, we cannot find anything to modify in the model that 
would make a significant difference in the selection of alternatives. A key attribute of a 
DSS is that its purpose is to assist in making a decision. Changes made to the model 
below a certain threshold would be counterproductive in that they could only serve in an 
attempt to make the model produce a decision. The real purpose of a DSS is to present the 
alternatives to the decision maker not to make the decision for him. 

If we had found some relationship displayed to us graphically which would appear 


unrealistic, the process of modifying it would be as follows: 


e Select the block in the hierarchy which represents the criteria that you wish to 
modify. 


e Modify the weighting for that criteria to a better representation of the model. 
e Display the results of the model and analyze it for reasonableness. 


e If the results are reasonable, the software has presented the user with a 
prioritized list of the alternatives. This list is shown graphically and a glance 
at it will show the user the best alternative to the worse alternative. A glance 
will also show the user the relative worth of each alternative. 
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e If the results are not reasonable or require investigation, then analyze the most 
significant contributors to those results in an attempt to discover the error in 
the model. Then return to the first step. 


OF SUMMARY 


The Critertum software meets the majority of user interface requirements for a 
DSS. Graphics displays, simple input devices and easily interpreted output are a 
requirement for a DSS to function at a high level of management. Without these 
attributes, the system would probably go unused or its output would acquire a poor 


reputation. 


54 


V. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 


The preceding chapters have demonstrated the utility of a DSS for military 
procurement. Large scale intricate decisions in procurement require detailed and nearly 
unlimited quantities of information to make the best possible decision. It is impossible for 
the decision maker to simplify, enumerate and analyze this information. Any decision 
maker must be able to champion his decisions to his superiors. To do this, he must, of 
course believe that he has made a prudent and reasonable decision. If he is overwhelmed 
with information that he cannot quantify, it will be difficult for him to believe that he is 
has covered every reasonable alternative. A DSS would assist the decision maker in 
quantifying the information coming to him. A DSS would also provide support to the 
decision maker in his belief that he has made the correct decision. Table 1 shows the 
differences between decisions made with the assistance of a DSS and more traditional 
methods. 


Non-DSS DSS Method 
Complex problems, dissimilar criteria, large Reduce complexity, use dissimilar criteria 


amounts of data comparison methods, automate data processing 


Time-consuming, laborious staff work Fast automated analysis 
No time or data for analysis Automated sensitivity analysis 


Inflexible response to changing requirements New insights, flexible data manipulation, system re- 


use, continuous process improvement 


Expensive acquisition process and weapons Improved control and cost performance 


Subjective, inconsistent decisions based solely on Consistent, objective decisions, open decision 


human judgment; subject to critics process; focus on process, not person 


“Fighting fires”; crisis management Quality analysis, planning, implementation 


Table 1. Benefits of DSS 
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Table 1 demonstrates the advantages of using a DSS for complex decisions. The 
decision maker not only makes a more reliable decision but receives a better 
understanding of the factors leading to that decision. 

A computer cannot replace human judgment or experience. The primary purpose 
of the DSS is to present the decision maker with information and analysis to augment 
these human qualities. The DSS will provide the user with analysis and probable results 
which will allow the decision maker to make a well-informed decision. 

From the survey, we found that the higher officer’s rank the less they tended to 
know about computers. But most of the officers surveyed believed that a working 
knowledge of computer systems would be necessary in the future. 

We found that 60% of the people responding believed there were problems with 
the present procurement system and over 75% of the respondents thought that the 
procurement system should be standardized. These respondents also thought that there 
should be a systematic approach to weapons system selection. Seventy-eight percent of 
the respondents believe that a DSS would help decision makers, improve efficiency and 
save money. They also believed that DSS would lead to a more objective decision. Of the 
respondents 49% responded that they did not understand what a DSS is. Of this 49%, 
60% were high ranking officers. This would point to a possible problem in the 
organization accepting DSS solutions for procurement. Further research and a much more 
detailed survey would be required to reach a conclusion relating to education and training 
for people at various levels in the procurement system. 

The Criterium software lacks some mechanisms that would make it more usable 
in large organizations. The most significant problem 1s its inability to communicate with 
other software, for example, database queries or an electronic mail system. Without these, 
the software would have difficulty in being accepted as an organizational tool. This could 
easily be overcome by simply modifying the software to work with another 
organizational tool which has these capabilities. For example, modern spreadsheets have 


all of these capabilities and are commonly used throughout most large organizations. 
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These modifications to work with other software used by the organization would also 
give the Criterium software the ability to query databases. 

The fact that the software lacks some mechanisms does not mean that it could not 
be used to introduce a pilot system and ultimately sell an organization on a larger DSS. 
The software will feed data to an Excel spreadsheet or text output to any other program; 
however, it only accepts data from other versions of itself. It would be possible for an 
organization to set up a hierarchy of DSS’ each feeding a file to the one at the next higher 
level in the organization resulting at the highest level in a final decision. The only 
mechanism that could successfully do this would be a network of computers that share 
files. This would be problematic in a military situation because of problems with 


classified material and information. 


B. RECOMMENDATIONS 


e The ROC should complete its efforts to standardize the military procurement 
system as soon as possible. In doing this, they should attempt to establish a 
DSS to support the decision making in the military procurement process. 


e The ROC should perform a broad and detailed survey to establish the factors 
that contribute to a decision of military procurement. This survey would 
establish a case for a DSS assisted decision making rather than the current 
practices. 


e Computers have an increasing presence at all management levels. The ROC 
should insure that every management level has a working understanding of 
computers and computer assisted decision making. 


e The ROC should try to move the procurement process from an individual 
decision to an organizational decision supported by analysis and computer 
assisted solutions. 


c. FUTURE DEVELOPMENTS 


In this thesis, has emphasized a DSS as a solution to military procurement 
problems. The ROC Navy’s procurement problems obviously cannot be solved by the 


implementation of one system. In order to meet its future needs, the Navy requires long- 


oO” 


range planning to provide database and networking support for a DSS. These database 


and networking systems must meet the needs of future support systems. 
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APPENDIX A. SURVEY 


1. Current procurement regulation are sufficient to prevent misconduct. (Circle one 
response) 


Strongly Disagree Disagree Do Not Know Agree Strongly Agree 


2. We should have a standardize, systematic approach to the weapons/equipment 
selection and acquisition. 


Strongly Disagree Disagree Do Not Know Agree Strongly Agree 


3. Should the following factors relating to weapons selection be included ? 


A. Feasibility of weapons/equipment acquisition (for instance the policy of 
the export country, export license; etc..) 


Strongly Disagree Disagree Do Not Know Agree Strongly Agree 
B. Cost 
Strongly Disagree Disagree Do Not Know Agree Strongly Agree 


C. Delivery Schedule 
Strongly Disagree Disagree Do Not Know Agree Strongly Agree 


D. Technology Transfer 


Strongly Disagree Disagree Do Not Know Agree Strongly Agree 


E. General Specifications of Weapons/Equipment 
Weapons Performance 
Integrated Logistics (ILS) 
Risk 
Strongly Disagree Disagree Do Not Know Agree Strongly Agree 
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F. Political Reasons 


Strongly Disagree Disagree Do Not Know Agree Strongly Agree 


G. Other (List Additional Factors) 











4. What would the distribution of each factor as a weighted value that would achieve 


the best possible results for a selection process? Please allocate the appropriate 
percentage relative to each factor, all percentages will sum to 100%. 


A. Feasibility of weapons/equipment acquisition (for instance the policy of 
the export country, export license; etc.) 





B. Cost 


C. Delivery Schedule 


D. Technology Transfer 
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E. General Specifications of Weapons/Equipment 


Weapons Performance 


Integrated Logistics (ILS) 


Risk 





Total = 100% 


F. Political Reasons 


G. Other (List Additional Factors) 














Total = 100% 


5. Would a decision support system (DSS) assist the decision maker to make the 
right choice 


Strongly Disagree Disagree Do Not Know Agree Strongly Agree 
6. If a DSS is adopted, will it help the process of weapons equipment acquisition to 
meet the following objectives: (Please circle all that apply) 
A. Efficiency related to time constraints 
B. Efficiency related to cost 


C. Efficiency related to combat effectiveness 


D. Objective 


E. Other (Please List) 











7. On which of the following factors would the effectiveness of a DSS rely ? (Please 
circle all that apply) 


A. The accuracy of budget estimates 
B. Pre-implementation preparatory staff work 
C. Implementation of contract 


D. Other (Please List) 











8. You are familiar with DSS 
Strongly Disagree Disagree Do Not Know Agree Strongly Agree 


APPENDIX B. RESULTS OF SURVEY 


Strongly i 1. Current procurement regulation are 


A gree sufficient to prevent misconduct. 


| Population 196 
Mean 2.8 


APOTee 





Do Not Know 


Disagree 





Strongly 
Disagree 





Respondents 


0 20 40 60 80 





Figure 26. Question 1. 
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Strongly Agree 





Population 197 
Mean 4.05 











Agree 
iM 2. We should have a standardize 
systematic approach to the 
weapon/equipment selection and 
Do Netnor acquisition. 
ml 
Disagree 





Strongly Disagree 





Respondents 
0 20 40 60 g0 100 120 
Figure 27. Question 2. 
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B3rF. 
M3SeE. 
3D. 
asc. 
w@ 3B. 
MSA. 


Strongly Agree 


Political Reasons 

General Specifications of Equipment 
Technology Transfer 

Delivery Schedule 

Cost 

Feasibility of Acquisition 














Agree 
Do Not Know 
Disagree 
Strongly 
Disagree Respondents 
0 10 20 30 40 


Figure 28. Question 3. 
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3.24 


3.92 
3./2 


Sr06 
3.14 
3.76 
Mean 


Population 100 





Should the following factors relating to 
weapons selection be included ? 


—— Se Cit e —— 


90 60 70 


Strongly Agree 





Agree 





Population 100 
| Mean 3.92 


Do Not Know 


| 5. Would a decision support system 
aie (DSS) assist the decision maker to make 
the right choice ? 






Disagree 


Strongly Disagree 


Respondents 





0 10 20 30 40 50 60 


Figure 29. Question 5. 
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6e. Other 


6d. Objectivity 


6c. Efficiency 
related to combat 


6b. Efficiency 
related to cost 


6a. Efficiency 
related to time 





6. If a DSS is adopted, will it help the 
process of weapons equipment 
acquisition to meet the following 
objectives ? 





Population 249 











Responses 
20 40 60 80 
Figure 30. Question 6. 
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100 


7. On which of the following factors would the 
effectiveness of a DSS rely ? 





{_-. Ss 


Implementation 
of contract 


Population 234 


Preparatory staff 
work 





The accuracy of 
budget estimates 





Responses 
0 20 40 60 80 100 
Figure 31. Question 7. 
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Strongly Agree 





8. Are you familiar with DSS ? 











| Population 100 
i Mean 3.25 
Disagree 
Strongly 
Disagree 


Respondents 


| 


! 


0 10 20 30 40 


Figure 32. Question 8. 
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APPENDIX C. CRITERIA 






























ie al. i 
10 Poiticat Considerations 









Level 2 Weights | Priorities Rating Set 


Technology Transfer | 60/ ——0.6 Knowledge and Skills 
 — e 40 0.4/Hardware and Software 
Risk 60 0.6|Delivery Time 


po OY 4 Technology Risk 
pS] 0.15 Weapons Availability 
p80} 0.8 International Trade Policies 
po) 0.15 Domestic Budget Considerations 
Political Considerations | 80| 0.8 Domestic Political Considerations 


20] 0.2 International Political Considerations 
Cost 30 0.3!Operating Cost 


po 700.7 | Procurement Cost 
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Knowedge and Skills Attermatives 

Missonefeaienss 00g 
eee ee 
i Paateg ——— 
integrated Logistics | Atermatives 
via Availability Atematives 

a a SS ce 
re 









































20 0.2! Training Cost 
s 0.05| Research and Development and Engineering Cost 


0.42) Initial Procurement 
ee 0.21 Initial Spare Parts and Training 
a 


Delivery Time 40, 0.17 
Technology Risk 
PayloadCapabiity. SS SC™~—ss|SCSC~<‘SC“‘CN’C#ON#(‘(#Uz«CG 
Financial Worthiness ofthe Contractor | OTC. 22 | 
international Trade Policies TC 0.17 
[Domestic Political Considerations | CBO CD. 
International PoliticalConsiderations | C“‘OT CD 
a 







12: 






Lowest Criteria _B-Korea Rating | B-Korea Priority 
Knowledge and Skills | 





0, 0 
Delivery Time | 100 0.42 
Technology Risk | 100] On3% 
Range | 20: 0.07 
a a en 
Integrated Logistics 30! 0.16 
Weapons Availability 20 0.08 
Financial Worthiness of the Contractor | 40) Oels 
International Trade Policies 100 0.29 
Domestic Budget Considerations | 100, 0.37 
Domestic Political Considerations | 10 0.04 
International Political Considerations | 30| 0.11 


Research and Development and Engineering Cost 


nitial Procurement 1 
Initial Spare Parts and Training 60 0.22 
Technical Support 20 0.08 


oO 
= 
i 
Co 


























Lowest Criteria _C-U.S. Rating | C-U.S. Priority 


Technology Risk cia 0:3 
[Domestic Budget Considerations TOT 
InitialProcurement 2 
0.3 
[Technical Support 831 





as 






T_T i 
Financial Worthiness ofthe Contractor 790] 0.88 
Domestic Budget Consieraions | of 03 
iniaProcurement 20 
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APPENDIX D. MODEL DEFINITION 


MODEL: Weapons System Selection 
Model Methodology: Analytical Hierarchy Process 


GOAL LEVEL: 
Weapons/Equipment Acquisition 
Notes: 
In choosing a weapons system source, our objective is to maximize the total 


utility of the weapons system and minimize the total cost. 


LEVEL 2: 

Technology Transfer _ 
Notes: 
This criterion indicates the technical capacity which will exist 
domestically after the procurement of the weapons system. The greatest 
advantage is represented by a purely domestic development and research 
followed by the relative willingness of the source country to supply 
technical expertise and information. 

Risk 
Notes: 
This criterion is used to show the risk of cost overruns, schedule 
slippage, or the failure to meet operational requirements, or possibly 


non-delivery. 


fs 


Combat Capability 


Notes: 
A figure that represents the performance characteristics of the weapons 
system evaluated on the same numeric scale as the other criteria. This 


is a composite of the capabilities of the weapons system. 


Feasibility of Weapons Acquisition 


Notes: 
The feasibility of acquiring weapons system, this factor involves trade 
restrictions, international embargoes, and domestic budget 


considerations. 


Political Considerations 


Cost 


Notes: 
This is the sum of all political considerations both domestic and 


international. 


Notes: 


This is the cost of both purchasing the system and operating it. 


LEVEL 3: 
Knowledge and Skills 


Notes: 
These are intangible skills and knowledge gained with the use and 


employment of advanced or new technologies. 


Hardware and Software 


Notes: 
This is the actual equipment or software gained and the ability to 


maintain and operate gained by the ROC. 
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Delivery Time 
Notes: 
Represents the time required to deliver an operational system, and 
possibly the time to train personnel in operation and maintenance of the 
system. 
Technology Risk 
Notes: 
This represents whether the weapon's system is a finished technology or 
a leading edge technology. This points to the risk that the weapon may 
in fact, be ineffective since it 1s untested in combat. 
Mission Effectiveness 
Notes: 
This is a reflection of the effectiveness of the weapons system. 
Integrated Logistics 
Notes: 
This represents whether the technology of the weapons system fits well 
into the existing systems used by the armed forces of the ROC. A weapons 
which uses unique spares or supplies would require unique logistics to 
maintain and use. 
Weapons Availability 
Notes: 
This is a reflection of the reliability and maintainability of the 


weapons system 


a 


Financial Worthiness of the Contractor 
Notes: 
This is the ability of the company producing the weapons system to 
produce the system even though it may not be to their financial 
advantage to do so. It also indicates the companies technology 
expertise to research and complete the project. 

International Trade Policies 
Notes: 
Some countries may have restrictions on the export of a particular type 
of weapons system, or they maybe an internationally recognized embargo 
on the country that is the source for the weapon. 

Domestic Budget Considerations 
Notes: 
Whether or not the legislative branch of government will budget money 
for this particular weapons system. 

Domestic Political Considerations 
Notes: 
Political considerations include domestic politics involved with 
government contracting. . 

International Political Considerations 
Notes: 
International politics involved with embargoes to outlawed countries or 


sources of supply which are not politically aligned with the ROC 
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Operating Cost 
Notes: 
Operating costs are the costs associated with the use of the equipment 
for example, steaming time required to train with the equipment, the 
cost resulting from a training exercise which expends ammunition, 
missiles, etc. 

Procurement Cost 
Notes: 
This 1s a cost of the equipment at initial purchase including the 
initial maintenance of the equipment and the initial parts support for 


the equipment. 


LEVEL 4: 
Range 
Notes: 
The range of the weapon 
Integration 
Notes: 
This reflects whether the weapons system is well integrated with the 
ship's existing weapons system. Does it provide extra capability or 
duplicate existing weapons systems. 
Payload Capability 
Notes: 
Does the weapon contain the capability of multiple types of warheads. Is 


the carrying capacity of the weapon suitable for the target. 
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Maintenance Spare Parts Cost 
Notes: 
This is the cost of the parts to maintain the equipment and the cost of 
preventative maintenance spares to keep the equipment at readiness. 
Training Cost 
Notes: 
The costs of training maintenance personnel and operations personnel. 
Research and Development and Engineering Cost 
Notes: 
This is the cost of the development of the weapons system and the cost 
of its production. 
Initial Procurement 
Notes: 
This is the minimum cost of acquiring a production run of the equipment. 
Initial Spare Parts and Training 
Notes: 
This is the cost of initial parts support and the initial training of 
personnel to operate and maintain the system. 
Technical Support 
Notes: 
This is the cost of technical personnel to support the weapons system 


while ROC personnel are being trained. 


ALTERNATIVES: 
A-France 

B-Korea 

C-U.S. 

D-ROC 
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